The present invention relates generally to the field of software development, and more particularly to software testing.
Software testing is an investigation conducted to provide information pertaining to the quality of the product or service under test. Software testing evaluates a program or application with the intent of finding errors or other defects (e.g., software bugs) by evaluating one or more properties for which the software is written to achieve. A test suite (e.g., collection of test cases and/or test scenarios) is developed for the purpose of validating a software program by demonstrating the software program meets a specified set of behaviors. Historically, the degree to which the software program is exercised by the test suite is measured as a percentage, based on one or more coverage criteria (e.g., function coverage, statement coverage, branch coverage, condition coverage, etc.). A program with high coverage is considered to be more thoroughly tested and have a lower chance of containing software bugs than a program with low coverage.
When designing test cases for software testing, multiple approaches are available, although primarily software testing methods include either white box testing or black box testing. In white box testing, the internal structure, design, and implementation of the software to be tested is known to a tester. For white box testing, the tester has access to the source code and chooses inputs to exercise code paths and determines the appropriate outputs. In black box testing, the software is a “black box” in which the internal structure, design, and implementation of the software is not known by the tester. For black box testing, the tester is only aware of external inputs and what an expected outcome should be but is not aware of how the program arrives at the expected outcome.
White box and black box testing include dynamic testing. Static testing is the implicit analysis of computer software that is performed without actually executing the program. Static testing involves individuals proofreading the source code (e.g., reviews, walkthroughs, inspections, etc.), utilizing programming tools/text editors to check the source code structure and/or, utilizing compilers (pre-compilers) to check syntax and data flow. Dynamic testing takes place when the program is actually running, thus allowing an examination of the physical response from the system to variables that are not constant and change with time. Dynamic testing involves manual and/or automated processes through unit tests, integration tests, system tests, and acceptance testing, etc., that execute specific test cases by providing input values and checking the output for anticipated results in order to determine success and/or failure.